1. Technical Field
The present invention relates generally to an improved data processing system, and in particular, to an improved method and apparatus for caching data in a memory.
2. Description of Related Art
Most early data processing systems consisted basically of a central processing unit, a main memory, and some sort of secondary input/output (“I/O”) capability. In these earlier systems, the main memory was the limiting element. Typically, the main memory was designed first and the CPU was then created to match the speed of the memory. This matching was performed to optimize the processing speed and is necessary even with today's high speed computers. Over time, logic circuit speeds increased along with the capacity requirements of main memory. With the need for increasing capacity in the main memory, the speed of the main memory could not keep up with the increasing speed of the CPU. Consequently, a gap developed between the main memory and the processor cycle time, which resulted in un-optimized processing speeds. As a result, a cache memory was developed to bridge the gap between the memory and the processor cycle time.
Using a cache to bridge the performance gap between a processor and main memory has become important in data processing systems of various designs from personal computers to work stations to data processing systems with high performance processors. A cache memory is an auxiliary memory that provides a buffering capability through which a relatively slow main memory can interface with a processor at the processor's cycle time to optimize the performance of the data processing system. Requests are first sent to the cache to determine whether the data or instructions requested are present in the cache memory. A “hit” occurs when the desired information is found in the cache. A “miss” occurs when a request or access to the cache does not produce the desired information. In response to a miss, one of the cache “lines” is replaced with a new one. The method to select a line to replace is called a replacement policy.
A number of different schemes for organizing a cache memory exist. For example, a fully associative mapping organization may be employed whereby a data address may exist in any location in the cache, or a direct mapping scheme may be employed in a cache memory whereby a data address may exist in only one location in the cache. A set associative scheme may be employed by partitioning the cache into distinct classes of lines, wherein each class contains a small fixed number of lines. This approach is somewhere between a direct mapped and a full associative cache. The classes of lines are usually referred to as “congruence classes.” The lines in a congruence class are usually referred to as sets (which indicate the number of locations an address can reside) in a congruence class in a set associative cache.
One generally used type of replacement policy is the least recently used (LRU) policy. An LRU policy is built upon the premise that the least recently used cache line in a congruence class is the least worthy of being retained. So, when it becomes necessary to evict a cache line to make room for a new one, an LRU policy chooses as a victim a cache line which is the least recently accessed set (or member) within a congruence class.
For an LRU policy, two types of operations must be carried out against the LRU state (which is maintained for each congruence class in a cache).
A most recently used-update (MRU-update) operation typically occurs due to a cache hit. It adjusts the LRU state such that the “hit” member is ordered ahead of all other members in that congruence class, establishing the cache line in that member position as the most worthy member in the congruence class.
A least recently used-victim-selection (LRU-victim-selection) operation typically occurs when a cache miss requires that a member be allocated to hold a cache line arriving from elsewhere in the storage hierarchy. The operation determines which cache line is the least worthy of being retained in the congruence class, evicts that cache line, and places the newly arriving cache line in its member position.
Often, favorable operating characteristics and reduced complexity implementations for a cache can be achieved when the victim selection and state update portions of a cache allocation policy are tightly integrated with a common pipeline for accessing the cache arrays, directory arrays, and allocation policy (e.g. LRU) state arrays.
Further, in such implementations, further benefits are typically derived when the victim selection occurs as early as possible in the common pipeline, and when for each operational use of the pipeline, at most one cache allocation policy state update is performed.
Those of ordinary skill in the art will recognize the following properties. The state information for the invalid member selection override exists in the directory. As caches grow larger (affecting size of data array, directory array, and LRU state array), the latency through a directory and to the victim selection logic grows relative to the latency through the LRU state array to the victim selection logic. Also, the circuit complexity (and latency) grows. Increases in the aggressiveness of processor operating frequencies compound this latency difference, making it more and more difficult to balance the desire to select a victim early with difference in latency between early arriving LRU state information and ever-later arriving invalid cache state information. With the currently available invalid cache line selection processes, the redirection or selection of an invalid cache line as a victim is not made quickly enough with respect to operating frequencies of the hardware. As a result, attempting to set an invalid cache line as a victim may not occur quickly enough with respect to the least recently used victim selection logic.
Therefore, it would be advantageous to have an improved method, apparatus, and computer instructions for marking or selecting an invalid cache line as a victim.